Unsupervised learning of visual invariance with temporal coherence
نویسندگان
چکیده
Natural scenes in a video stream contain rich collections of visual transformations. In this paper, a generic neural network is built to learn visual invariance from videos in an unsupervised manner. We use temporal coherence to learn both visual transformations and features with complex invariances. Without fine-tuning with labels, our invariant features are superior for classifying objects in still images. The learned features out-perform features learned with sparsity in vision benchmarks Caltech-101, STL-10 and COIL-100.
منابع مشابه
Cs229 Final Report: Unsupervised Learning of Temporally Coherent Features for Action Recognition
Human vision is remarkably well-adapted to object and action recognition tasks. Without any thought, we can identify a given object regardless of its position in our field of vision or its specific orientation. Our visual system isolates characteristics that define a particular object, or even a particular action, while being invariant to noise. Recent literature suggests that at least some of ...
متن کاملUnsupervised Learning of Visual Representations using Videos
This is a review of unsupervised learning applied to videos with the aim of learning visual representations. We look at different realizations of the notion of temporal coherence across various models. We try to understand the challenges being faced, the strengths and weaknesses of different approaches and identify directions for future work. Unsupervised Learning of Visual Representations usin...
متن کاملLearning to Categorize Objects Using Temporal Coherence
The invariance of an objects' identity as it transformed over time provides a powerful cue for perceptual learning. We present an unsupervised learning procedure which maximizes the mutual information between the representations adopted by a feed-forward network at consecutive time steps. We demonstrate that the network can learn, entirely unsupervised, to classify an ensemble of several patter...
متن کاملTemporal Coherence, Natural Image Sequences, and the Visual Cortex
We show that two important properties of the primary visual cortex emerge when the principle of temporal coherence is applied to natural image sequences. The properties are simple-cell-like receptive fields and complex-cell-like pooling of simple cell outputs, which emerge when we apply two different approaches to temporal coherence. In the first approach we extract receptive fields whose outpu...
متن کاملUnsupervised Natural Visual Experience Rapidly Reshapes Size-Invariant Object Representation in Inferior Temporal Cortex
We easily recognize objects and faces across a myriad of retinal images produced by each object. One hypothesis is that this tolerance (a.k.a. "invariance") is learned by relying on the fact that object identities are temporally stable. While we previously found neuronal evidence supporting this idea at the top of the nonhuman primate ventral visual stream (inferior temporal cortex, or IT), we ...
متن کامل